1. Field of the Invention
The present invention relates to video communication and more particularly to a method and an apparatus for mixing bit streams of compressed video from more than one video source.
2. Description of the Prior Art
Video communication between more than two video terminals often requires a Multipoint Control Unit (MCU), a conference controlling entity that typically is a piece of equipment located in a node of a network or in a terminal which receives several channels from access ports and, according to certain criteria, processes audiovisual signals and distributes them to a set of connected channels. Examples of MCUs include the MGC-100, which is available from Polycom Networks Systems Group, the assignee of the present invention. A terminal (which may be referred to as an endpoint) is a location on the network, capable of providing real-time, two-way audio and/or visual communication with other terminals or the MCU.
The MCU may include a bank of decoders, encoders, and bridges. The MCU may use a large amount of processing power to handle video communications between a variable number of participants, using a variety of communication and compression standards and a variety of bit streams, for example. The MCU may need to compose these bit streams into a single output stream that is compatible with a conference participant to which the output stream is being sent. Thus, an MCU may need to convert a variety of compression and communication standards having a variety of resolutions into an output stream of a single resolution, compression standard, and communication standard. Consequently, an MCU is typically an expensive and rather complex product.
In more recent years, videoconferencing and other forms of multimedia communications have become more commonplace. The advent of personal computers having videoconferencing capabilities creates a demand for MCUs having the capability of multimedia communication between devices housed on Local Area Networks (LANs). This trend raises the need for low cost MCUs. However, low cost MCUs may only handle a limited multipoint communication (e.g., a limited number of compression standards, a limited number of conferees, and a limited number of layouts).
U.S. Pat. No. 5,675,393 discloses an image processing apparatus for composing a plurality of coded images into one image without decoding the plurality of coded images when the images are transmitted using the H.261 standard.
Quarter Common Intermediate Format (QCIF) is a videoconferencing format that specifies a video frame containing 144 lines and 176 pixels per line. This is one-fourth the resolution of Common Intermediate Format (CIF). QCIF support is required by some of the International Telecommunications Union (ITU) videoconferencing standards.
U.S. patent application Ser. No. 09/768,219, Pub. No. 2001/0019354A1, entitled, “Method and an Apparatus for Video Mixing of Bit Streams,” discloses a method and apparatus for mixing as many as four QCIF H.263 compressed video bit streams into a composite CIF image.
However, U.S. patent application Ser. No. 09/768,219 does not teach how an MCU can handle a conference in a real communication environment where the different terminals transmit their intra frames at different times forcing the MCU to multiplex an intra frame from one terminal with an inter frame from other terminals. An inter frame is an example of a referential frame, which is a frame that gives the difference between two frames. An intra frame is a type of non-referential frame. A frame is composed of an array of MBs (Macro Blocks). In H.263 it is difficult to multiplex a low-resolution intra frame as part of a high-resolution inter frame without converting it to an inter frame format, since the encoding of an intra-MB in an intra frame is different from an intra-MB in an inter frame.
Furthermore, U.S. patent application Ser. No. 09/768,219 does not teach how the MCU handles a need to request an intra frame from one of or some of the sources. For example, it is not clear how the MCU handles a request for an intra frame, which is due to a packet from one of the sources that is missing or arrives late.
In addition, the method that is described in U.S. patent application Ser. No. 09/768,219 requires recalculation of the quantizers and/or the Discrete Cosine Transform (DCT) coefficients in some of the MBs. This recalculation requires time and computing power. Therefore, (although possibly not recognized heretofore) there is a need for a method and apparatus that can handle a conference in a real communication environment by composing low-resolution compressed video frames (such as QCIF and CIF frames) into a higher resolution frame such as a CIF or 4CIF frame, respectively, without recalculating the Motion Vector Data (MVD) and/or the quantization coefficients.